23 research outputs found
Rotating Features for Object Discovery
The binding problem in human cognition, concerning how the brain represents
and connects objects within a fixed network of neural connections, remains a
subject of intense debate. Most machine learning efforts addressing this issue
in an unsupervised setting have focused on slot-based methods, which may be
limiting due to their discrete nature and difficulty to express uncertainty.
Recently, the Complex AutoEncoder was proposed as an alternative that learns
continuous and distributed object-centric representations. However, it is only
applicable to simple toy data. In this paper, we present Rotating Features, a
generalization of complex-valued features to higher dimensions, and a new
evaluation procedure for extracting objects from distributed representations.
Additionally, we show the applicability of our approach to pre-trained
features. Together, these advancements enable us to scale distributed
object-centric representations from simple toy to real-world data. We believe
this work advances a new paradigm for addressing the binding problem in machine
learning and has the potential to inspire further innovation in the field
Complex-Valued Autoencoders for Object Discovery
Object-centric representations form the basis of human perception and enable
us to reason about the world and to systematically generalize to new settings.
Currently, most machine learning work on unsupervised object discovery focuses
on slot-based approaches, which explicitly separate the latent representations
of individual objects. While the result is easily interpretable, it usually
requires the design of involved architectures. In contrast to this, we propose
a distributed approach to object-centric representations: the Complex
AutoEncoder. Following a coding scheme theorized to underlie object
representations in biological neurons, its complex-valued activations represent
two messages: their magnitudes express the presence of a feature, while the
relative phase differences between neurons express which features should be
bound together to create joint object representations. We show that this simple
and efficient approach achieves better reconstruction performance than an
equivalent real-valued autoencoder on simple multi-object datasets.
Additionally, we show that it achieves competitive unsupervised object
discovery performance to a SlotAttention model on two datasets, and manages to
disentangle objects in a third dataset where SlotAttention fails - all while
being 7-70 times faster to train
PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers
Time-dependent partial differential equations (PDEs) are ubiquitous in
science and engineering. Recently, mostly due to the high computational cost of
traditional solution techniques, deep neural network based surrogates have
gained increased interest. The practical utility of such neural PDE solvers
relies on their ability to provide accurate, stable predictions over long time
horizons, which is a notoriously hard problem. In this work, we present a
large-scale analysis of common temporal rollout strategies, identifying the
neglect of non-dominant spatial frequency information, often associated with
high frequencies in PDE solutions, as the primary pitfall limiting stable,
accurate rollout performance. Based on these insights, we draw inspiration from
recent advances in diffusion models to introduce PDE-Refiner; a novel model
class that enables more accurate modeling of all frequency components via a
multistep refinement process. We validate PDE-Refiner on challenging benchmarks
of complex fluid dynamics, demonstrating stable and accurate rollouts that
consistently outperform state-of-the-art models, including neural, numerical,
and hybrid neural-numerical architectures. We further demonstrate that
PDE-Refiner greatly enhances data efficiency, since the denoising objective
implicitly induces a novel form of spectral data augmentation. Finally,
PDE-Refiner's connection to diffusion models enables an accurate and efficient
assessment of the model's predictive uncertainty, allowing us to estimate when
the surrogate becomes inaccurate.Comment: Project website: https://phlippe.github.io/PDERefiner
Mesh Neural Networks for SE(3)-Equivariant Hemodynamics Estimation on the Artery Wall
Computational fluid dynamics (CFD) is a valuable asset for patient-specific
cardiovascular-disease diagnosis and prognosis, but its high computational
demands hamper its adoption in practice. Machine-learning methods that estimate
blood flow in individual patients could accelerate or replace CFD simulation to
overcome these limitations. In this work, we consider the estimation of
vector-valued quantities on the wall of three-dimensional geometric artery
models. We employ group-equivariant graph convolution in an end-to-end
SE(3)-equivariant neural network that operates directly on triangular surface
meshes and makes efficient use of training data. We run experiments on a large
dataset of synthetic coronary arteries and find that our method estimates
directional wall shear stress (WSS) with an approximation error of 7.6% and
normalised mean absolute error (NMAE) of 0.4% while up to two orders of
magnitude faster than CFD. Furthermore, we show that our method is powerful
enough to accurately predict transient, vector-valued WSS over the cardiac
cycle while conditioned on a range of different inflow boundary conditions.
These results demonstrate the potential of our proposed method as a plugin
replacement for CFD in the personalised prediction of hemodynamic vector and
scalar fields.Comment: Preprint. Under Revie
Mesh convolutional neural networks for wall shear stress estimation in 3D artery models
Computational fluid dynamics (CFD) is a valuable tool for personalised,
non-invasive evaluation of hemodynamics in arteries, but its complexity and
time-consuming nature prohibit large-scale use in practice. Recently, the use
of deep learning for rapid estimation of CFD parameters like wall shear stress
(WSS) on surface meshes has been investigated. However, existing approaches
typically depend on a hand-crafted re-parametrisation of the surface mesh to
match convolutional neural network architectures. In this work, we propose to
instead use mesh convolutional neural networks that directly operate on the
same finite-element surface mesh as used in CFD. We train and evaluate our
method on two datasets of synthetic coronary artery models with and without
bifurcation, using a ground truth obtained from CFD simulation. We show that
our flexible deep learning model can accurately predict 3D WSS vectors on this
surface mesh. Our method processes new meshes in less than 5 [s], consistently
achieves a normalised mean absolute error of 1.6 [%], and peaks at 90.5
[%] median approximation accuracy over the held-out test set, comparing
favourably to previously published work. This demonstrates the feasibility of
CFD surrogate modelling using mesh convolutional neural networks for
hemodynamic parameter estimation in artery models.Comment: (MICCAI 2021) Workshop on Statistical Atlases and Computational
Modelling of the Heart (STACOM). The final authenticated version is available
on SpringerLin
Meta-learning for fast cross-lingual adaptation in dependency parsing
Meta-learning, or learning to learn, is a technique that can help to overcome
resource scarcity in cross-lingual NLP problems, by enabling fast adaptation to
new tasks. We apply model-agnostic meta-learning (MAML) to the task of
cross-lingual dependency parsing. We train our model on a diverse set of
languages to learn a parameter initialization that can adapt quickly to new
languages. We find that meta-learning with pre-training can significantly
improve upon the performance of language transfer and standard supervised
learning baselines for a variety of unseen, typologically diverse, and
low-resource languages, in a few-shot learning setup
Toetsing van de Groene Weide Meststof in de praktijk : Demovelden van de gebiedsgerichte pilot Kunstmestvrije Achterhoek, 2018
The aim of the project Biobased Fertilisers Achterhoek (in Dutch Kunstmestvrije Achterhoek) project is to make fertilisation practice more sustainable by means of the use of locally available nutrients from renewable sources. The project is part of the sixth action program of the Netherlands serving the Nitrate Directive. One of the objectives is to identify the eligible product quality and product composition of fertilising products from animal manure and sludge which can be produced by means of best available techniques for manure and sludge processing. This objective has been worked out by WUR-Wageningen Environmental Research (WUR-WENR) in a monitoring program. A research topic is testing of a new fertilising product from animal manure and other (most renewable) nitrogen sources in demonstration field trials. This document reports the first results of the year 2018. The demonstration field trials will be continued in 2019 and 2020